Polymorphic path planning for robotic devices

ABSTRACT

Provided is a robot-implemented, real-time, process to plan a coverage path, the process including: obtaining environment-sensor data indicating distances from the robot to surfaces in a portion of a working environment; obtaining odometry-sensor data; based on the environment-sensor data and the odometry-sensor data, determining at least a part of a coverage path of the robot through the working environment; and commanding an electric-motor driver to move the robot along the at least part of the path.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of provisional patent application62/535,555, filed on Jul. 21, 2017.

In this patent, certain U.S. patents, U.S. patent applications, or othermaterials (e.g., articles) have been incorporated by reference.Specifically, U.S. patent application Ser. Nos. 62/618,964, 62/573,591,62/613,005, 62/583,070, 15/272,752, 62/661,802, and 62/666,266 arehereby incorporated by reference. The text of such U.S. patents, U.S.patent applications, and other materials is, however, only incorporatedby reference to the extent that no conflict exists between such materialand the statements and drawings set forth herein. In the event of suchconflict, the text of the present document governs, and terms in thisdocument should not be given a narrower reading in virtue of the way inwhich those terms are used in other materials incorporated by reference.

FIELD OF INVENTION

The present disclosure relates to robotic devices, and moreparticularly, to path planning for robotic devices.

BACKGROUND

Autonomous and semi-autonomous robotic devices are increasingly usedwithin consumer homes and commercial establishments. Such devices mayinclude robotic cleaners, such as vacuum cleaners, lawn mowers,weed-removers, gutter cleaners, and mops. During operation, theserobotic devices typically follow movement paths within the workingenvironment while executing their task.

Embodiments of the present invention provide a path planning method forrobotic devices responsive to stimulus from a configuration space.

SUMMARY

The following presents a simplified summary of some embodiments of thepresent inventions. This summary is not an extensive overview of theinvention. It is not intended to identify key or critical elements or todelineate the scope of the inventions. Its sole purpose is to presentsome embodiments in a simplified form as a prelude to the more detaileddescription that is presented below.

Some aspects include a robot-implemented, real-time, process to plan acoverage path, the process including: obtaining, with one or moreprocessors of a robot, environment-sensor data indicating distances fromthe robot to surfaces in a portion of a working environment of the robotfrom sensors carried by the robot; obtaining, with one or moreprocessors of the robot, odometry-sensor data indicating changes inposition of the robot over time; based on the environment-sensor dataand the odometry-sensor data, determining, with one or more processorsof the robot, at least a part of a coverage path of the robot throughthe working environment, wherein determining the at least part of thecoverage path comprises determining lengths of segments of the coveragepath, the segments having a linear or arc motion trajectory, and thesegments forming a zig-zag pattern, a boustrophedon pattern, or a spiralpattern that covers at least part of the working environment; andcommanding, with one or more processors of the robot, an electric-motordriver to move the robot along the at least part of the path.

The features and advantages described in the specification are notexclusive and, in particular, many additional features and advantageswill be apparent to one of ordinary skill in the art in view of thedrawings, specification, and claims. Moreover, it should be noted thatthe language used in the specification has been principally selected forreadability and instructional purposes and should not be read aslimiting.

BRIEF DESCRIPTION OF DRAWINGS

Non-limiting and non-exhaustive features of the present inventions aredescribed with reference to the following figures, wherein likereference numerals refer to like parts throughout the various figures.

FIGS. 1A-1D illustrate an example of an encoder system implemented witha robotic device wheel, in accordance with some embodiments.

FIG. 2A and FIG. 2C illustrate a top view of boustrophedon route planwith preprogrammed properties for elements forming the route plan forworking environments of the same shape but different size, in accordancewith some embodiments.

FIG. 2B and FIG. 2D illustrate a top view of a route plan withproperties for elements forming the route plan determined at run timefor working environments of the same shape but different size, inaccordance with some embodiments.

FIG. 3A and FIG. 3C illustrate a top view of boustrophedon route planwith preprogrammed properties for elements forming the route plan forworking environments of similar size but different shape, in accordancewith some embodiments.

FIG. 3B and FIG. 3D illustrates a top view of a route plan withproperties for elements forming the route plan determined at run timefor working environments of similar size but different shape, inaccordance with some embodiments.

FIG. 4A, FIG. 4C, and FIG. 4E illustrate a top view of boustrophedonroute plan with preprogrammed properties for elements forming the routeplan for working environments of different shapes and sizes, inaccordance with some embodiments.

FIG. 4B, FIG. 4D, and FIG. 4F illustrate a top view of a route plan withproperties for elements forming the route plan determined at run timefor working environments of different shapes and sizes, in accordancewith some embodiments.

FIG. 5 illustrates a top view of a route plan skewed with respect to theperimeters of an environment, in accordance with some embodiments.

FIG. 6A and FIG. 6B illustrate an initial route plan and updated routeplan after discovery of a new area, in accordance with some embodiments.

FIGS. 7A-7E illustrate predicted wheel radii parameters andcorresponding error in pose for augmented Kalman filter and tracingerror gradient descent methods, in accordance with some embodiments.

FIGS. 8A-8C illustrates examples of position prediction methods withimplemented RNNs, embodying features of the present invention.

FIG. 9 is a schematic diagram of an example of a robot with which thepresent techniques may be implemented.

FIG. 10 is a flowchart describing embodiments of a path planning method,embodying features of the present invention.

DETAILED DESCRIPTION OF CERTAIN EMBODIMENTS

The present techniques will now be described in detail with reference toa few embodiments thereof as illustrated in the accompanying drawings.In the following description, numerous specific details are set forth inorder to provide a thorough understanding. It will be apparent, however,to one skilled in the art, that the present techniques may be practicedwithout some or all of these specific details. In other instances, wellknown process steps and/or structures have not been described in detailin order to not unnecessarily obscure the present techniques.

Some embodiments include a robot configured to implement a path planningmethod that is responsive to stimulus from an observed environment usingone or more processors of the robotic device. Some embodiments segment aworking environment into regions and then dynamically adjust a coveragepattern within each of those regions abased on sensed attributes of theenvironment. In some embodiments, a topological graph represents theroute plan of a robotic device and is described with a set of verticesand edges, the vertices being linked by edges. Vertices may berepresented as distinct points while edges may be lines, arcs or curves.The properties of each vertex and edge may be provided as arguments atrun-time based on real-time sensory input of the environment. Thetopological graph may define the next actions of the robotic device asit follows along edges linked at vertices. While executing the movementpath, in some embodiments, rewards may be assigned as the robotic devicetakes actions to transition between states and uses the net cumulativereward to evaluate a particular movement path comprised of actions andstates. A state-action value function may be iteratively calculatedduring execution of the movement path based on the current reward andmaximum future reward at the next state. One goal is to find optimalstate-action value function and optimal policy by identifying thehighest valued action for each state. As different topological graphsincluding vertices and edges with different properties are executed overtime, the number of states experienced, actions taken from each state,and transitions increase. The path devised by one or more processors ofthe robotic device iteratively evolves to become more efficient bychoosing transitions that result in most favorable outcomes and byavoiding situations that previously resulted in low net reward. Afterconvergence, the evolved movement path is determined to be moreefficient than alternate paths that may be devised using real-timesensory input of the working environment.

In embodiments, the properties of the vertices and edges of thetopological graph describing the movement path of the robotic device maybe provided at run-time as an argument based on sensory input of therobotic device. A property of a vertex may be, for example, its positionand the number and position of vertices linked via edges. A property ofan edge may be, for example, edge type such as a line or arc, edgelength or radius depending on edge type, angular orientation andconnecting vertices. In some embodiments, vertices and edges may alsoinclude other properties such as floor type, room identifier and/orpossible operations (e.g., mopping, sweeping, UV, etc.). In embodiments,the topological graph may be implemented within a taxicab coordinatesystem, where the path is limited to following along the grid lines ofthe taxicab coordinate system, thereby limiting edge type to a line. Inother embodiments, the number of roots or nodes of the topological graphis limited to one. One or more processors of the robotic devicedesignate a vertex as a root within the topological graph that may becapable of reaching the whole graph from the designated vertex, i.e.there is a path from the root to all other vertices and edges within thegraph.

In embodiments, one or more processors of the robotic device collectsensory input of the environment and create a map of the environment bystitching newly collected readings with previously collected readings.Techniques for creating a map from sensory input may be found in U.S.Patent App. No. 62/618,964, U.S. Patent App. No. 62/613,005, and U.S.Patent App. No. 62/573,591, the entirety of the contents of which areincorporated by reference above. As the one or more processors receivesensory input, in some embodiments, they create a representation of themap in a taxicab coordinate system and devise a topological path withindiscovered areas, i.e. areas for which sensory input has been collected,the edges of the path being lines following along the gridlines of thetaxicab coordinate system. Sensory input may include, for example, acollection of depth measurements. In embodiments, depth measurements ofthe working environment (e.g., from the perspective of a robot) may betaken using depth measurement devices such as LIDAR, camera, laser,sonar, ultrasonic, stereo vision, structured light vision devices orchip-based depth sensors using CMOS or CCD imagers, IR sensors and such.In embodiments, other sensory input may be used, for example, dataindicating flooring type or obstacle detection. For example, inembodiments, the one or more processors may receive input from opticalfloor sensors upon detection of a pattern of reflected light emittedonto the floor. The one or more processors may use one or more stages ofsignal processing and machine learning to determine to a degree ofcertainty the type of floor upon which the robotic device moves. As afurther example, in embodiments, an obstacle sensor may detect obstaclesor clutter on the floor based on reflection of emitted light. Theobstacle sensor may then transmit that sensory input to the one or moreprocessors of the robotic device for further processing. In embodiments,the one or more processors of the robotic device may receive input fromtactile sensors when physical contact is made with an object.

The devised topological path may be based on estimates of suitableproperties for vertices and edges based on sensory input received. Thenext action or movement of the robotic device may be along a pathdefined by the estimated properties of the vertices and edges. As therobotic device executes the action, it transitions from its currentstate to a new state. After completing each action and transitioning toa new state, in embodiments, a reward may be assigned and a state-actionvalue function may be iteratively calculated based on the current rewardand the maximum future reward at the next state. In some embodiments,e.g., where time is not considered discrete, the value of the reward maybe dependent on sequential time required to complete the action andtransition to the new state, where a greater negative reward is assignedfor longer times. As such, in some embodiments, the robotic deviceincurs a negative reward at all times. Since the robotic device ispenalized for time, any event that may reduce the efficiency of therobotic device in terms of cleaning time increases its overall penalty.These events may include collisions with obstacles, number of U-turns,repeat coverage, transitions between different types of flooring, andswitching rooms. In embodiments, these events may be directly used toassign negative reward thereby acting as optimization factorsthemselves. In embodiments, other efficiency metrics may be used, suchas coverage. Once the robotic device completes its task and hence thetopological movement path required to complete the task, a positivereward value (e.g., predetermined or dynamically determined) may beassigned. A net reward value for the executed movement path, consistingof a sequence of states and actions, may then be calculated as the sumof the cumulative negative reward from the multiple actions taken whiletransitioning from one state to another and the positive reward uponcompletion of the task.

As multiple work sessions are executed over time, in embodiments,optimal state-action value function and optimal policy from whichactions from different states are selected may be determined. From asingle state, there may be several actions that can be executed. Thesequence of states and actions that result in the maximum net reward, insome embodiments, provides the optimal state-action value function. Theaction from a state which results in the highest reward provides theoptimal policy for the given state. As different movement paths areexecuted over time, the number of states experienced, actions taken fromeach state, and transitions increase.

In some embodiments, the processor(s) devises a path for the roboticdevice iteratively over multiple work sessions, evolving to become moreefficient by choosing transitions that result in most favorable outcomesand by avoiding situations that previously resulted in low net reward.In embodiments, the processor(s) selects properties for each movementpath within an assigned work cycle such that the cumulative penaltyvalue for consecutive work cycles have a lowering trend over time. Insome embodiments, the processor(s) of the robotic device may executeconvergence to a particular movement path when the reward is maximizedor a target reward is achieved or a period of time has passed afterwhich the processor(s) may converge the movement path to the path withthe highest reward. After convergence, assuming the system did not fallinto a local minimum or is able to get out of a local minimum, theprocessor(s) may deem the evolved movement path to likely be moreefficient than alternate paths that may possibly be devised usingreal-time sensory input of the working environment. In some embodiments,the processor(s) may avoid falling into a local minimum using techniquessuch as random restarts, simulated annealing and tabu search. Forexample, in employing random restarts technique, the processor(s) mayrandomly restart the process of searching for a candidate solutionstarting at a new random candidate after a certain amount of time, whilestill saving in memory previous candidate solutions. In embodimentswherein simulated annealing technique is used, the processor(s) replacesa current candidate solution when a better solution is found but mayalso probabilistically replace the current candidate solution with aworse solution. In embodiments using tabu search technique, theprocessor(s) may not return back to recently considered candidatesolutions until they are sufficiently in the past. This is expected toprovide a more reliable and efficient method for a robotic device todevise path plans as their movements are evaluated and optimized inreal-time, such that the most efficient movements are eventuallyexecuted and factors reducing efficiency, including but not limited to,repeat coverage, collisions with obstacles, transitions betweendifferent types of flooring and U-turns, are reduced with thefine-tuning of properties over time.

Accordingly, some embodiments implement methods for optimizing pathplanning of a robotic device by responding to stimulus from an observedenvironment. Different movement paths may be experimented and evaluatedwithin a particular working environment such that the most efficientmovement path for the robotic device may evolve out of the process. Insome embodiments, the working environment may be all areas of a space ormay be an area within the space, such as a room. In some embodiments, auser, using a user graphical interface (see, e.g., U.S. application Ser.Nos. 15/272,752, 62/235,408, and 62/272,004, incorporated herein byreference) or a programming interface of the robotic device or otherinput device, or a processor(s) of the robotic device may assign aunique tag to different working environments. The unique tag may be usedby the processor(s) to separate data relating to different environments.In embodiments wherein different working environments of the roboticdevice comprise connected rooms within a space, a room graph may becreated by the processor wherein each working environment may be definedseparately in order to avoid or minimize such issues. The room graph maybe a mathematical graph that has nodes representing the rooms andvectors determining how the rooms are connected to one another. Eachroom may have its own properties associated with it such as a centroid,a set of perimeter points and doors corresponding to the workingenvironment, doors labelled with the room to which they lead to, aunique number and/or label to identify the room and flags signifyingvisitations to the room. In embodiments, a topological graph may be usedto describe the movement path of the robotic device. The topologicalgraph consists of vertices and edges, the vertices being linked to oneanother by edges. Vertices are represented as distinct points whileedges may be lines, arcs, or curves. The processor of the robotic devicemay employ topological geometry to spatially relate objects. Inembodiments, the processor of the robotic device may use topologicalgeometry to perform transformation of objections, such as, translation,rotation, reflection, stretching, bending and twisting, butneighborhoods, i.e. spatial relations, may remain preserved. Forexample, a circular curve centered within a larger circular curvecontains a point P between the two circular curves and a point Q withinthe smaller circular curve. After transformation, the smaller circularcurve has been stretched and bent to become a rectangular slit butremains within the larger circular curve. To preserve neighborhoods, thepoint P must remain between the two curves while the point Q must remainwithin the inner curve.

The topological graph described herein is similar to a Euclidean graph,such that the movement path described by the graph consists of a set ofvertices and edges. However, in a Euclidean graph the edges are limitedto being lines and the lines connecting vertices are equal to theEuclidean distance. This means the path between two vertices is alwaysequal to the shortest path between them. In topological geometry, theedge may be a line, arc, or curve, hence the path between two verticesmay not necessarily be the shortest path as in Euclidean geometry.Further, with topological graph, the elements of the graph, namelyvertices and edges, may be deformed by means of variation in assignedproperties. In embodiments, the properties of the vertices and edges ofthe topological graph describing the movement path of the robotic deviceare provided at run-time as an argument based on sensory input of therobotic device. This allows the processor of the robotic device to adaptits path to the observed environment. A property of a vertex may be, forexample, its position, such as angular orientation, and the number andposition of vertices linked via edges. A property of an edge may be, forexample, edge type such as a line or arc, edge length or radiusdepending on edge type, angular orientation and connecting vertices.With topological geometry, any movement path may be devised with pathelements, such as vertices, edges, and their associated properties. Forexample, a boustrophedon movement path, characterized by back and forthmovement, may be considered equivalent to a set of vertices linked byedges, the vertices having properties defining position and angularorientation of linked vertices and the edges having properties definingedge type, such as a line, with given length, angular orientation andconnecting vertices. As a further example, a spiraling movement path maybe defined by a set of vertices linked by edges having edge typeproperty of an arc, the radius of the arc increasing linearly at eachstep to achieve the spiraling movement.

In embodiments, the dimensionality of the topological graph describingthe movement path is reduced by implementing it within a taxicabcoordinate system. In taxicab geometry, all paths follow along gridlinesof the coordinate system, thereby limiting edge type to a line. Further,the distance metric between vertices is the rectilinear distance or L1norm, mathematically represented by:

${d\left( {p,q} \right)} = {{{p - q}} = {\sum\limits_{i = 1}^{n}{{p_{i} - q_{i}}}}}$

where (p,q) are vectors p=(p₁, p₂, . . . , p_(n)) and q=(q₁, q₂, . . . ,q_(n)). With taxicab geometry, the rectilinear distance between the twopoints is independent of the structure of the path following along thegridlines of the taxicab coordinate system. In some embodiments, theprocessor of the robotic device begins to collect sensory input of theenvironment and create a map by stitching newly collected readings withpreviously collected readings. In embodiments, sensory input is assumedto be independent and identically distributed (IID), where eachobservation has the same probability distribution as all otherobservations and all observations are mutually independent. Ifobservations are defined to assume values in if

⊆R, then two random variables x and Y are identically distributed if andonly if:

P[x≥X]=P[x≥Y],∀x∈

and are independent if and only if:

P[y≥Y]=P[y≥Y|x≥X]∧P[x≥X]=P[x≥X|y≥Y],∀x,y∈

In embodiments, the sensory input may go through various layers ofmathematical processing, such as feature scaling, Bayesian probabilisticmethods, and the like. Sensory input may include depth measurements orother measurements from which depth of objects may be inferred, such astime-of-flight or pixmap. Sensing devices include, but are not limitedto, LIDAR, camera, laser, sonar, ultrasonic, stereo vision, structuredlight vision or chip-based depth sensors using CMOS or CCD imagers, IRsensors, and the like. Methods for creating a map from sensory input maybe found in U.S. Patent App. No. 62/618,964, U.S. Patent App. No.62/573,591, and U.S. Patent App. No. 62/613,005, the contents of whichare incorporated by reference above. In some embodiments, as theprocessor(s) receives sensory input, it creates a representation of themap within a taxicab coordinate system and begins to devise atopological path within discovered areas, i.e. areas for which sensoryinput has been collected. For example, the processor(s) may mark asperimeter certain vertices of the taxicab coordinate system based onsensory input of the environment and the topological path may berestricted to remain within the marked perimeters. The topological pathmay be devised by estimating properties of the vertices and edges thatdefine the path based on real-time sensory input. For implementationwithin a taxicab coordinate system, the edges of the path are linesfollowing along gridlines of the coordinate system and are linked atvertices. As the robotic device begins to move by following along thedevised topological path, the processor (used interchangeably with“processor(s)”) of the robotic device continues to receive sensoryinput. The processor may use the sensory input to revise and expand themap as well as revise the properties of vertices and edges defining thetopological path. As more data is collected a better perception of theenvironment is revealed and the map becomes more accurate and inclusiveof the area. The topological path may consist of any number of verticesand edges, depending on the shape, size, etc., of the area discovered,and may be arranged in any number of ways. Because of the stochasticnature of the work place and partial observability, despite the effortsof processor to propose an optimal path, there may exist better pathswhich were not obvious to the robot at the time of decision making.However, over time the topological path is optimized by, for example,combining vertices by passing or eliminating an edge, removing or addingvertices, and/or edges and changing the direction or position ofvertices and/or edges. In embodiments, an optimized path achieves bettercoverage of the discovered area. In embodiments, the robotic device maybegin to start coverage of the working environment and performing workbefore exploration of the entire area is complete. In such cases, theprocessor is likely to choose movements that are locally optimal but notglobally optimal.

The next action or movement of the robotic device is along the pathdefined by the estimated properties of the vertices and edges chosenbased on real-time sensory input. As the robotic device executes theaction, it transitions from its current state to a new state andmovement from one state to the next is defined by a discrete time slot.This may be represented by a Markov Chain comprised of a sequence ofrandom variables s₁, s₂, s₃, . . . . The random variables are states therobotic device may experience and form a set S called the state space.The topological graph defining the movement path of the robotic devicemay therefore be thought of as a sequence of states s∈S, where statesare connected by paths and are each defined with a discrete time stampt∈T. For the robotic device to transition from a current state s to nextstate s′, the robotic device performs an action a∈A over a time span oft to t′, displacing a distance d along an edge of the topological graph.When the state space is defined by a taxicab coordinate system, thedistance d is given by the rectilinear distance or L1 norm anddisplacement is along a line. For a Markov chain, having Markovproperty, the probability of moving to a next state is dependent only onthe present state. This is mathematically represented by P(s′|s). AMarkov chain may, therefore, be represented by a topological graph,where the edges of graph t are labelled by the probabilities oftransitioning from one state at time t to another at time t′. A Markovchain may be further extended to a Markov Decision Process (MDP) throughthe addition of actions (choices) and rewards (motivation), such thatthere are multiple actions that may be chosen from a single state and adifferent reward associated with each action. MDP is a five-tuplecomprising a finite set of states S, a finite set of actions A, theprobability that action a will lead to state s′ at time t′ given byP(s′|s), the immediate reward after transitioning from state s to states′ given by r, and the discount factor γ, representing the difference inimportance between future and present rewards. The goal of MDP is tofind an optimal policy function π that specifies the highest rewardedaction a to take for each state s. For a MDP, after completing eachaction and transitioning to a new state, a reward is assigned and astate-action value function is iteratively calculated as the expectedvalue of the current reward plus the discounted maximum future reward atthe next state. The state-action value function provides the value of astate. The processor of the robotic device does not require anyvisualization in choosing the next action of the robotic device, it onlyinvolves, in some embodiments, optimization of the state-action valuefunction. In optimizing the state-action value function, the highestrewarded actions from each state are concurrently (e.g., simultaneously)identified and used in deriving the optimal policy. In embodiments,where the time is not considered discrete, the value of the reward maybe dependent on sequential time required to complete the action andtransition to a new state, where a greater negative reward is assignedfor longer times. In such a case, the robotic device is always incurringnegative reward and actions having smaller negative reward areconsidered superior. (Of course, the selection of sign is arbitrary, andembodiments may also implement the reverse arrangement, which is not tosuggest that any other description is limiting.). Events that increasethe time required to complete an action and transition to the next statemay therefore indirectly increase the amount of negative rewardincurred. Other optimization factors may also assign negative reward,including but not limited to, collisions with obstacles, number ofU-turns, repeat coverage, transitions between different types offlooring or switching rooms. Once the robotic device completes its task,and hence the movement path required to complete the task, the processormay assign a predetermined positive reward. A net reward value for theexecuted movement path, consisting of a sequence of states and actions,in embodiments, may then be calculated as the sum of the cumulativenegative reward from the multiple actions taken while transitioning fromone state to another and the positive reward upon completion of thetask.

Over time, the goal is to find optimal state-action value function andoptimal policy from which actions from different states are selected.For a single state, there may be several actions that can be executed.The sequence of states and actions that result in the maximum net rewardprovide the optimal state-action value function for a given state. Theaction for a given state that results in maximum reward provides theoptimal policy for the given state. An optimal policy for a state spacemay then contain the highest valued action corresponding to multiplestates. As different movement paths are executed over time, the numberof states experienced, actions taken from each state, and transitionsincrease. The path devised by the processor of the robotic device mayiteratively evolve to become more efficient by choosing transitions thatresult in most favorable outcomes and by avoiding situations whichpreviously resulted in low net reward. After convergence, assuming thesystem did not fall into a local minimum or is able to get out of alocal minimum, the evolved movement path is trusted to be more efficientthan alternate paths which may be devised using real-time sensory inputof the working environment. In order to get out of local maximin,stochastic optimization is employed. This provides a reliable andefficient method for a robotic device to devise path plans as theirmovements are evaluated and optimized in real-time such that the mostefficient movements are eventually executed and factors reducingefficiency, including but not limited to, repeat coverage, collisionswith obstacles, transitions between different types of flooring, andU-turns, are reduced with the fine-tuning of properties over time.

The Markov Decision Process (MDP) consisting of a sequence of states andactions followed by rewards is mathematically notated below. Actions aretaken to transition from one state to another and after transitioning toeach new state a reward is assigned. For a sequence of states andactions, the net reward is the sum of rewards received for the sequenceof states and actions, with future rewards discounted. The expected netreward for the execution of a sequence of states and actions is given bya state-action value function. The goal is to find an optimalstate-action value function by identifying sequence of states andactions with highest net reward. Since multiple actions can be takenfrom each state, the goal is to also find an optimal policy thatindicates the action from each state with the highest reward value.Consider a sequence of states s and actions a followed by rewards r:

s _(t) ,a _(t) ,r _(t+1) ,s _(t+1) ,a _(t+1) ,r _(t+2) ,s _(t+2) ,a_(t+2) ,r _(t+3) , . . . a _(T) ,r _(T) ,s _(T)

The net return R_(T) to be expected in the future is the sum of therewards received for the sequence of states and actions beginning fromstate s_(t) and ending with terminal state s_(T). This is mathematicallyrepresented by:

R _(T) =r _(t+i)γ¹ r _(t+2)+ . . . +γ^(T−t−1) r _(T)

where 0≤γ<1 is a discount factor applied as distant rewards are lessimportant. The value of a state-action pair Q (s,a) is defined asequivalent to the expected return R_(T) for the sequence of states andactions beginning with state s_(t) and action a_(t) and ending withterminal state s_(T).

Q(s,a)=E[R _(T) |s _(t) =s,a _(t) =a]

By finding the sequence of states and actions which maximize thestate-action value function Q (s,a), the optimal value function Q* (s,a)is identified:

Q*(s,a)=maxE[R _(T) |s _(t) =s,a _(t) =a]

And the optimal policy for each state can be derived by identifying thehighest valued action which can be taken from each state.

π*(s)=argmax Q*(s,a)

To iteratively calculate the state-action value function for a givenstate s and action a, the Bellman Optimality equation may be applied.The optimal value function obeys Bellman Optimality equation and can beexpressed as:

Q*(s,a)=E[r+γmaxQ*(s′,a′)]

The equation expresses that the value for a given state s and action ashould represent the current reward r observed at state s plus themaximum discounted γ future reward for the next state s′ the roboticdevice would end up in. This equation can be used to iterativelycalculate the state-action value for a given state s and action a as thesequence of states and action are executed. i is the iteration numberand begins at i=0, with Q₀(s′,a′) being initially assumed based, forexample, on previous experience, the midpoint of the min and max valuepossible, or an arbitrary value.

Q _(i+1)(s,a)=E[r+γmaxQ _(i)(s′,a′)]

Based on the definition of an expected value, the equation is equivalentto:

Q _(i+1)(s,a)=ΣP(s′|s)[r+γmaxQ _(i)(s′,a′)]

where P(s′|s) is the probability that action a will lead to state s′, aspreviously described above. In embodiments, the sequence of states andactions corresponds to the states visited and actions taken whileexecuting the movement path from start to finish, where actions aredefined by the properties of vertices and edges chosen based on sensoryinput of the robotic device. Over time, as more states are visited anddifferent actions from each state are evaluated the system will convergeto find the most optimal action to take from each state thereby formingan optimal policy. Further, as different sequences of states andactions, i.e. movement paths, are evaluated over time, the system willconverge to the most optimal sequence of states and actions.

In some embodiments, the robotic device evaluates different movementpaths while offline (e.g., between work sessions, such as betweencleaning sessions, like while charging) using sensory input of theworking environment previously collected and stored in memory of, orotherwise accessible to, the robotic device. Or in some cases, suchprocessing may be offloaded to a remote application, e.g., a processorin a charging state or cloud-based infrastructure. In some embodiments,the robotic device experiments with (e.g., simulates and determinesoutcomes from) previously executed and new movement paths. Properties ofvertices and edges are inferred from previously collected sensory input.In some embodiments, the robotic device is able to enhance and fine-tunemovement paths while offline (or some embodiments may perform theseactions online). The estimated time required to complete a task (e.g.,cleaning a room with greater than threshold area coverage) is used tocalculate a theoretical net reward value. The movement path with thegreatest theoretical net reward value may be executed at the nextcleaning cycle and based on measured performance (e.g., time to clean orcoverage) the true net reward value may be determined. Some embodimentsmay determine a difference between estimated and measured performanceand adjust model parameters to reduce the difference.

For the robotic device to physically take action and move, the processormay actuate the wheels, tracks, or other actuated interfaces with theenvironment. This may be accomplished, in some embodiments, throughthree subsystem layers of the processor, which in some cases, is onboardthe robot. In embodiments, the first subsystem layer is the velocitycontroller, which receives requested linear and angular velocities anddisplacement from the polymorphic navigation algorithm (e.g., in theprocessor, implementing the techniques above) after the next action ofthe robotic device is chosen. The velocity controller may set the linearand angular velocity in m/s and rad/s, respectively. Formally, a linearvelocity in the x-direction of a coordinate system is represented byV_(x) while an angular velocity is represented by V_(w). The velocitycontroller may also be used to monitor the set velocity to increase thelikelihood that the target value is reached and maintained and to readand return the linear and angular velocities from a platform layer. Thisfirst subsystem layer, in some embodiments, also comprises an emergencystop function, such that the velocity is set to 0 m/s in the case of anemergency. Further, the ramp up/down time for a desired speed can be setwithin the velocity controller, thereby controlling acceleration anddeceleration of the robotic device. The gradual acceleration anddeceleration protects the motor and gears as a sudden increase in speedimposes a large torque on the wheel motors thereby causing wear to themotor and gears. For an emergency situation, ramp down is set to 0 m/s,causing the robotic device to immediately stop.

At the second layer, in some embodiments, a differential drivecontroller may be responsible for converting velocity set in thevelocity controller into actual velocity. The linear and angularvelocity set by the velocity controller must be translated into avelocity for each wheel. The differential drive controller sets thevalues on each of the individual motors and at this layer polarityindicates direction. The third layer is the embedded motor driver.Details of its functions are hidden from higher level subsystems, suchas the velocity controller and differential drive controller. Thisdriver controls the direction that the motor spins by setting a value of0, 1, or −1, where for example, 0 indicates no rotation, 1 indicatesclockwise rotation, and −1 counterclockwise rotation. At an even lowerlevel, the direction the motor spins can be controlled by applying avoltage of 0V, 5V or −5V to a general-purpose input/output (GPIO) pin onthe integrated circuit (IC) or controller chip. The embedded motordriver also controls each motor individually by sending pulses ofvoltage to each motor. The number of voltage pulses per second controlsthe rotational speed of the motor while the value of voltage pulsecontrols the direction of rotation of the motor. Initially equal numberof voltage pulses per second are sent to each of the motors of therobotic device. Since the motor is an analogue device and smallvariations exist in their wiring the number of rotations of each motorwill not be exactly the same for every voltage pulse received. The gearsand gear box also introduce some noise as they are slightly differentfrom one another. Further, slippage adds to the unpredictability of thespeed and/or displacement of each wheel. Therefore, the number ofvoltage pulses per second needs to adjusted based on such noise in orderto achieve the target rotational speed and displacement over a period oftime.

In some embodiments, the processor in each of the three layers describedabove has three modes: regular operational mode, in which the controllerwill accept velocity commands and check for safety events; safety mode,in which a safety event has occurred and the robotic device remainsstopped until the event is acknowledged by the application layer; andrecovery mode, in which a safety event is acknowledged by theapplication layer and corrective action is taken or the safety event isignored. The three modes may have a mutex lock in relation to oneanother such that the robotic device cannot move if any of theprocessors of the three layers are in safety or recovery mode.

The distance traveled can be measured with a variety of techniques,e.g., by visual odometry or by an encoder. As shown in FIGS. 1A and 1B,an encoder wheel 100 has square slots 101 that, when aligned with LEDtransmitter 102 and LED receiver 103, allow IR light transmitted by LEDtransmitter 102 to be received by LED receiver 103. LED transmitter 102and receiver 103 are stationary, while encoder wheel 100 rotates whenpower is provided to motor 104 shown in FIG. 1C. In FIG. 1D motor 104interfaces with wheel 105 of the robotic device through gears containedwithin enclosure 106, such that rotation of the motor causes rotation ofencoder wheel 100 and wheel 105 of the robotic device. As encoder wheel100 rotates, IR light transmitted by LED transmitter 102 is able to bereceived by LED receiver 103 each time square slot 101 is in line withIR transmitter 102 and receiver 103. The number of times IR light isreceived by receiver 103 is counted, where each time receiver 103receives IR light is referred to as a tick. In embodiments, the numberof times the IR light is blocked may alternatively be counted. Thenumber of slots 101 on encoder wheel 100 define the resolution of theencoder, a constant referred to as encoder resolution. The encoderresolution can be used to specify the amount of encoder wheel rotationgiven the number of times IR light is received. The amount of rotationof encoder wheel 100200 is correlated with the amount of displacement ofwheel 104 of the robotic device on the working surface. Therefore, giventhe number of slots 101 on encoder wheel 100, the number of tickscounted, and the ratio of size between encoder wheel 100 and wheel 104of the robotic device, the distance traveled can be determined. Further,given the time, the speed may also be determined. A tick resolution maybe defined as the amount of distance traveled between two counted ticksor two instances of receiving light by the IR receiver. In order tocalculate the total distance traveled the total number of ticks can bemultiplied by the tick resolution. In addition to the encoder system, agyroscope, such as L3GD20 gyroscope by STMicroelectronics, may also beused. The gyroscope may use an I²C (inter-integrated-circuit) interfacewith two pins or an SPI (serial peripheral interface) with four pins tocommunicate with the processor.

Due to imperfection in analog motors, gears, tiny spikes in voltage,measurement errors and such, a difference between the desired traveleddistance and the actual traveled distance is expected. When the moduleimplementing the polymorphic algorithm above determines the next action,in some embodiments, the corresponding linear and angular velocities anddisplacement requested to achieve said action is passed from thevelocity controller, to the differential driver controller, then to theembedded motor driver to actuate movement of the wheels and complete theaction. The traveled distance measured by the encoder may notnecessarily be the same as the desired target displacement. In someembodiments, an adaptive processor is used to record the differencebetween the target value and actual value of the displacement over onetime step. This is considered the absolute error and is given by:

error=|target value−actual value|

As the robotic device moves, the absolute error sum may also becalculated by summating the absolute error for each time step:

${{error}\mspace{14mu} {sum}} = {\sum\limits_{t = 1}^{\infty}{error}_{t}}$

In some embodiments, the processor of the robotic devices uses a controlloop feedback mechanism is used to minimize the difference between thetarget value and actual value by correcting the future number of voltagepulses provided to each motor based on previous results, wherein thenumber of voltage pulses per second controls the rotational speed of themotor and hence measured displacement over one time step. Inembodiments, the future number of voltage pulses provided is correctedby using a proportional adjustment. For example, if a wheel is receiving100 pulses per second and previously measured displacement is tenpercent more than the target displacement desired, a proportionaladjustment P is applied to the future number of voltage pulses such that90 pulses per second are provided in order to attempt to achieve thetarget displacement. This can be mathematically represented by:

P=K _(p)*error

where K_(p) is the proportional gain constant. This helps smoothen thetrajectory of the robotic device, however since the adjustment isapplied at a time when the wheel is already faster than desired, theinitial velocity of the wheel prior to the adjustment still has animpact on the trajectory which is affected by the original overshoot. Anintegral I of past errors over time may be applied as a furthercorrection to eliminate residual error. This is mathematicallyrepresented by:

I=K _(i)∫₀ ^(t)errordt

where K_(i) is the integral gain constant. The integral can becalculated by summating the absolute error for each time step over aperiod of time. The integral correction helps reduce systematic errors,such as errors created due to, for example, a wheel being slightlylarger or a motor being slightly more powerful or a motor receivingslightly higher voltage than expected. The integral may have a limit,where only a limited portion of the history is considered. A derivativeD may also be calculated and used in applying a correction to thevariable controlling the target value in order to reduce the error andis given by:

$D = {K_{d}\frac{\Delta \; {error}}{\Delta \; {time}}}$

where K_(d) is the derivative gain constant. The derivative is the bestestimate of the future trend of the error based on its current rate ofchange. The three constants K_(p), K_(i), and K_(d) may be tuned to thespecific application such that the difference between the target valueand actual value is minimized. The proportional, integral and derivativecorrections may be combined to produce an output that may be applied asa correction to the variable controlling the desired outcome in order toreduce the overall error.

output=P+I+D

In this case, for example, the correction may be applied to the numberof voltage pulses per second provided to the motor in order to achievethe desired displacement and thereby reduce the error between target andactual displacement. At startup, the accumulated error is reduced by thegradual acceleration of the robotic device. This allows the displacementand corresponding adjustment of the motor speed to be applied before therobotic device reaches maximum speed resulting in smaller displacementswhile only limited feedback is available.

The implementation of a feedback processor is beneficial in some casesas a differential drive mechanism, comprised of two independently drivendrive wheels mounted on a common axis, used by robotic devices may behighly sensitive to slight changes in velocity in each of the wheels.The small errors in relative velocities between the wheels can affectthe trajectory of the robotic device. For rolling motion the roboticdevice rotates about an instantaneous center of curvature (ICC) locatedalong the common axis. To control the trajectory of the robotic devicethe velocities of the two wheels can be varied. The angular velocity ωabout the ICC can be related to the velocities ν_(l) and ν_(r) of theleft and right wheels, respectively, by:

${\omega \left( {R + \frac{l}{2}} \right)} = v_{r}$${\omega \left( {R - \frac{l}{2}} \right)} = v_{l}$

where l is the length of the axle connecting the two wheels and R is thedistance from the ICC to the midpoint of the axle connecting the twowheels. If ν_(l)=ν_(r), then there is only forward linear motion in astraight line. If ν_(l)=−ν_(r), then the ICC is at the midpoint of theaxle and there is only rotation in place. If

${v_{l} = {0\frac{m}{s}}},$

then the ICC is at the left wheel, i.e. rotation is about the leftwheel. The same applies for the right wheel if

$v_{r} = {0{\frac{m}{s}.}}$

To navigate me robotic device, assume the robotic device centered at themidpoint between the two wheels and is at a position (x,y), headed in adirection θ with respect to the horizontal x-axis. By adjusting ν_(l)and ν_(r) the robotic device may move to different positions andorientations. The position of the ICC can be found by:

ICC=[ICC_(x),ICC_(y)]=[x−R sin θ,y+R cos θ]

At time t+δt the pose of the robotic device (x′,y′,θ′) is:

$\begin{bmatrix}x^{\prime} \\y^{\prime} \\\theta^{\prime}\end{bmatrix} = {{\begin{bmatrix}{\cos \left( {\omega \; \delta \; t} \right)} & {- {\sin \left( {\omega \; \delta \; t} \right)}} & 0 \\{\sin \left( {\omega \; \delta \; t} \right)} & {\cos \left( {\omega \; \delta \; t} \right)} & 0 \\0 & 0 & 1\end{bmatrix}\begin{bmatrix}{x - {ICC}_{x}} \\{y - {ICC}_{y}} \\\theta\end{bmatrix}} + \begin{bmatrix}{ICC}_{x} \\{ICC}_{y} \\{\omega \; \delta \; t}\end{bmatrix}}$

For a differential drive the navigation strategy of the robotic deviceis to move in a straight line, rotate in place, then move in a straightline again in order to reach desired (x,y,θ). For motion in a straightline where ν_(l)=ν_(r)=ν, the equation used to determine the pose of therobotic device reduces to:

$\begin{bmatrix}x^{\prime} \\y^{\prime} \\\theta^{\prime}\end{bmatrix} = \begin{bmatrix}{x + {v\; \cos \; \theta \; \delta \; t}} \\{y + {v\; \sin \; \theta \; \delta \; t}} \\\theta\end{bmatrix}$

And for rotation in place where ν_(l)=−ν_(r), the equation used todetermine the pose of the robotic device reduces to:

$\begin{bmatrix}x^{\prime} \\y^{\prime} \\\theta^{\prime}\end{bmatrix} = \begin{bmatrix}x \\y \\{\theta + \frac{2v\; \delta \; t}{l}}\end{bmatrix}$

In some embodiments, an H bridge IC or driver, such as Quadruple Half-Hbridge driver SN754410 by Texas Instruments or other similar bridgedrivers, may be used to control DC motors. The H bridge is used to drivethe motor's direction and regulate its speed. For example, QuadrupleHalf-H bridge driver SN754410 has 16 pins and is able to enable a pairof DC motors on each side of the IC using pins 1 and 9. These pins willrun the motors by enabling them with a voltage generated by a batteryconnected to pin 16. The left motor leads connect to output pins 3 and 6and right motor leads to output pins 11 and 14. Input pins 2, 7, 10, and15 may be connected to a Beaglebone Black Board (BBB) from which inputcommands are sent. A BBB is a low-power open-source single-boardcomputer. Pins 4, 5, 12 and 13 are connected to ground.

FIG. 2A illustrates a preprogrammed boustrophedon route plan withpredetermined properties of vertices and edges defining the route plan.The route is shown (here and in other similar figures) in a plan view ofa working environment. For example, length 200, a property of edge 201,is predetermined. FIG. 2B illustrates an example of a boustrophedonroute plan with length 202, a property of edge 203, is determined at runtime in response to the observed environment. Edges may also be referredto as segments of a path between decision points. FIG. 2B illustratesthe advantages of a polymorphic navigational approach, whereinproperties of vertices and edges forming a route plan are determined atrun time, instead of a route plan with preprogrammed properties as inFIG. 2A. Some embodiments may implement a hybrid approach in whichportions of a route are pre-programmed and others are determined at runtime (e.g., while performing a task in a working environment by arobot). It can be seen, that even for a basic rectangular workingenvironment, the boustrophedon route plan with preprogrammed length 200in FIG. 2A does not provide the most efficient movement path for therobotic device. In comparison, boustrophedon route plan in FIG. 2B withedge length 202 determined at run time has half as many U-turns therebydecreasing the total working time of the robotic device and improvingits efficiency. In the polymorphic approach, the robotic device is ableto decide the properties of vertices and edges at run time based on thereal-time input of the working environment (e.g., by sensing dimensionsof a room). In this case, length 202 of edge 203 in FIG. 2B is chosen tobe the length of the working environment, thereby resulting in lessU-turns and allowing for a more efficient route plan for the roboticdevice. Further, the number of vertices and edges are determined atrun-time based on the real-time input of the working environment makingit more likely that the entire working environment is covered.

FIG. 2C illustrates a boustrophedon route plan with preprogrammed length200, a property of edge 201, while FIG. 2D illustrates an embodiment ofa boustrophedon route plan with length 202, a property of edge 203,determined at run time. The working environment illustrated in FIGS. 2Cand 2D are of the same shape as the working environments in FIGS. 2A and2B, but of different size. Assuming the same robotic device is used inFIGS. 2A and 2C, the boustrophedon route plan in FIG. 2C has samepreprogrammed length 200 as in FIG. 2A, despite the increase in size inthe working environment. Regardless of the size and shape of the workingenvironment, the properties of vertices and edges forming theboustrophedon route remain the same as they are predetermined andpreprogrammed into the processor(s) of the robotic device. However, withthe polymorphic navigation approach, properties of vertices and edgesare determined at run time and are based on the real-time input of theworking environment. In this case, with a larger working environment,length 202 of the boustrophedon route plan in FIG. 2D is greater as itis chosen to be the length of the working environment, allowing forsignificantly less U-turns thereby improving the efficiency of therobotic device. The number of back and forth movements is also increasedin this larger working environment through the addition of addedvertices and edges such that the entire working area may be covered. Inthis way, the number and properties of vertices and edges may be chosensuch that the route plan is adapted to the real-time workingenvironment, wherein different number and properties of vertices andedges forming a route plan are chosen for working environments in FIGS.2B and 2D by the same robotic device.

FIG. 3A illustrates a boustrophedon route plan with preprogrammed length200 of edge 201, and preprogrammed angular orientation 300 of vertex 301relative to vertex 302. FIG. 3B illustrates an embodiment of aboustrophedon route plan with length 202 of edge 203 and angularorientation 303 of vertex 304 relative to vertex 305 determined at runtime. A more efficient route plan is devised and executed in FIG. 3B asa polymorphic navigational approach is used wherein properties ofvertices and edges forming the boustrophedon route plan are determinedat run time based on the real-time input of the working environment.Edge 203 spans the length of the room, thereby reducing the number ofU-turns and improving the efficiency of the robotic device in coveringthe working area. In FIG. 3A, boustrophedon route plan withpreprogrammed properties 200 and 300 results in increased U-turns asparameters are predetermined, remaining the same for any workingenvironment.

As shown in FIGS. 3C and 3D, the working environment illustrated is ofsimilar size as the working environment in FIGS. 3A and 3B, but ofdifferent shape. Assuming the same robotic device is used in FIGS. 3Aand 3C, the boustrophedon route in FIG. 3C has the same preprogrammedproperties 200 and 300 as in FIG. 3A, despite the change in shape of theworking environment. Regardless of the size and shape of the workingenvironment, the properties of vertices and edges associated with theboustrophedon route plan remain the same as they are predetermined andpreprogrammed into the processor of the robotic device. However, withthe polymorphic navigational approach, properties of vertices and edgesconnected in forming the boustrophedon route plan are adapted to theworking environment. In this case, length 200 of edge 201 and angularorientation 300 of vertex 301 relative to vertex 302 in FIG. 3D arechosen based on the observed working environment to devise a moreefficient route plan for the robotic device. In comparison to FIG. 3Cwhere properties of vertices and edges are preprogrammed, thepolymorphic approach allows for significantly less U-turns therebyimproving the efficiency of the robotic device and increased coverage ofthe working area. In this way, properties are chosen such that theboustrophedon route plan is adapted to the real-time workingenvironment, wherein different properties of vertices and edges arechosen for working environments in FIGS. 3B and 3D by the same roboticdevice.

FIG. 4A-F illustrate boustrophedon route plan with preprogrammedproperties for vertices and edges (A, C, E) and an embodiment ofboustrophedon route plan with properties determined at run time (B, D,F) for working environments of different sizes and shapes. Referring toFIG. 4A, boustrophedon route plan with same preprogrammed properties 200and 300 as previously described, are used in navigating the workingenvironment. An unnecessary number of U-turns are used and areas remainuncovered by the robotic device as properties of vertices and edgesforming the path are predetermined independently of the shape and sizeof the working environment. Referring to FIG. 4B wherein properties ofvertices and edges are dependent on the real-time working environmentand are determined at run time, the route plan, with route parameterssuch as edge lengths 400, 401, 402 and angular orientation betweenvertices 403 and 404, improves efficiency of the robotic device andcoverage of the area. Referring to FIGS. 4C and 4E, boustrophedon routeplan with preprogrammed properties 200 and 300 remains the same despitethe change in size and shape of the working environment, assuming thesame robotic device is used as in FIG. 4A. Referring to FIGS. 4D and 4Fin comparison, assuming the same robotic device is used as in FIG. 4B,properties of vertices and edges such as edge lengths 405, 406 or 408,409, 410 and angular orientation between vertices 407 or 411 in additionto the number of vertices and edges are chosen such that the route planis adapted to the working environment when a polymorphic navigationalapproach is used. This allows for the robotic device to devise a routeplan that may improve its working efficiency by, for example, reducingthe number of U-turns and increasing the total area covered.

In embodiments, the route plan is devised within an area smaller thanthe total area perceived. A margin with respect to the perimeter of theworking environment is set and the route plan is devised within the areabounded by the set margins. In embodiments, the margins are set by theprocessor of the robotic device based on observed input of theenvironment while in another embodiment the size of margins arepredetermined. In another embodiment, the margins are set by the userthrough a graphical user interface (GUI), such as those noted above.Margins minimize disruption during execution of route plans that mayarise due to inaccuracies in measurements or measurements collected bylow resolution devices where the perceived perimeter may not be exact.If the route plan happens to intersect with the true perimeter, theroute plan is disrupted. Margins also help accommodate irregularities inthe shape of the environment, where, for example, it would be hard toaccommodate a boustrophedon pattern if there is small alcove locatedalong one of the walls of the room. In such a case where margins areused, the small alcove would be left out of the covered area. Inembodiments, after the route plan devised within the area bordered bythe margins is complete, the robotic device covers the areas between theperimeter and margins thereby coverings areas previously left out. Thisensures the robotic device cleans in a methodical way while covering allareas. Without margins, it is difficult to find a non-repeating andoptimal route plan, while with margins a repeating route plan may beexecuted within the margins followed by coverage along the perimeter.

In some embodiments, a graphical user interface (GUI), such as onepresented on a smartphone, computer, tablet, dedicated remote control,or any device that may display output data from the robotic device andreceive inputs from a user may be used (see, e.g., U.S. application Ser.No. 15/272,752, and 62/661,802, incorporated herein by reference) toallow the user may choose to only clean areas within the borders of themargin and skip coverage along the perimeter. A user may such GUI tochoose this if the perimeters are difficult to cover, for example, ifthere is minimal space between a piece of furniture and the perimeter.In some embodiments, the user may choose to cover only certain areasalong the perimeter, thereby avoiding difficult areas to cover. In someembodiments, the user may choose to cover areas bordered by the marginsin one work session and complete coverage along the perimeter in aseparate work session. Various alternating combinations are possible.For example, the user may only choose along the perimeter if it requiresmore cleaning than central areas within the margins and vice versa.

In embodiments, the map of the area, including but not limited to (whichis not to imply other descriptions are limiting) perimeters, doorways,sub areas, perimeter openings, and information such as coverage pattern,room tags, order of rooms, etc. may be available to the user through theGUI (see, e.g., U.S. patent application Ser. No. 15/272,752,incorporated herein by reference). Through the GUI, a user may review,accept, decline, or make changes to, for example, the environmentalrepresentation and settings and operations of the robotic device withinthe environment, which may include, but are not limited to, route plan,type of coverage algorithm of the entire area or each subarea,correcting or adjusting map boundaries, order of cleaning subareas,scheduled cleaning of the entire area or each subarea, and activating ordeactivating tools such as UV light, suction and mopping. User inputsare sent from the GUI to the robotic device for implementation. Data maybe sent between the robotic device and the user interface through one ormore network communication connections. A variety of types of wirelessnetwork signals may be used, including, but not limited to, radiosignals, Wi-Fi™ signals, or Bluetooth™ signals.

In some embodiments, a route plan is devised within discovered areas ofthe environment where it is not required that the entire workingenvironment be mapped before devising a route plan. In some embodiments,observations of the environment continue while the robot executes aroute plan within discovered areas, resulting in newly discovered areasand a more defined perceived environment. In embodiments, the roboticdevice executes a route plan based on the perceived environment anddiscovery of new areas alternate. For example, the robotic device mayfirst be in discovery mode where observations of the environment arecollected thereby discovering new areas. Following discovery mode, therobotic device may then execute a route plan devised within thediscovered areas based on the discovered areas perceived. Inembodiments, the robotic device concurrently (e.g., simultaneously)devises and executes a route plan based on the perceived environment anddiscovers new areas. For example, the processor of the robotic devicemay perceive an area surrounding its starting point and devise a routeplan within the perceived area. While it executes the route plan, newareas are perceived for which a second route plan is devised. Inembodiments, the route plan may be altered or amended as new areas arediscovered and perceived. In other embodiments, a route plan iscompleted without alteration and a second route plan is devised withinnewly discovered areas.

In embodiments, after executing a first route plan within a firstdiscovered area, the robotic device moves to a central or preferredlocation within the second discovered area before beginning to deviseand execute a new route plan within the second discovered area. Inembodiments, the robotic device devises the second route plan within thesecond discovered area such that the ending point of the first routeplan is close to or is the same as the starting point of the secondroute plan. In environments where this is difficult to achieve, therobotic device moves to a central or other preferred location within thesecond discovered area and starts the second route plan from there.

In embodiments, the route plan devised may not be aligned with theperimeters of the environment. The route plan devised using thepolymorphic navigational approach may be skewed with respect to theperimeters of the environment. For example, in FIG. 5 segments of routeplan 500 are skewed with respect to perimeter 501 of environment 502.

In some embodiments, the route plan devised and executed by the roboticdevice may comprise coverage of central areas within the environmentfirst, followed by coverage around the perimeters of the environment.For example, in FIG. 5 route plan 500 may first be executed withincentral areas of environment 502 followed by route plan 503 alongperimeter 501 of environment 502. In other embodiments, route plan 503along perimeter 501 of environment 502 may be executed first beforeexecuting route plan 500 covering central areas of environment 502. Insome embodiments, only central areas of the environment may be coveredor perimeters of the environment may be covered.

In some use cases, e.g., where the environment comprises multiple rooms,a route plan covering the central areas followed by the perimeters ofeach room in sequence may be executed. For example, for an environmentcomprised of three rooms a route plan covering the central area then theperimeter of the first room, followed by the central area then theperimeter of the second room and finally the central area then theperimeter of the third room may be executed. In some embodiments,central coverage of all rooms may be executed first, then coverage ofthe perimeters of all rooms and vice versa. For example, a route plancovering, in the following order, the central area of the first room,the central area of the second room, the central area of the third room,the perimeter of the third room, the perimeter of the second room andthe perimeter of the first room may be executed. Or, for example, aroute plan covering, in the following order, the perimeter of the thirdroom, the perimeter of the second room, the perimeter of the first room,the central area of the first room, the central area of the second roomand the central area of the third room may be executed. The sequence ofcoverage of the central areas and perimeters of the rooms may bedependent on the layout of the rooms within the environment. The mostefficient sequence of coverage of the central areas and perimeters ofthe rooms may be determined over time using, for example, an MDPtechnique.

In embodiments, at least one distance sensor, such as a TOF, IR, ortouch sensor, disposed along the side of the robotic device may be usedto estimate the distance of the robot from an object, such as a wall orobstacle. For example, it may be desirable for the robot to maintain aparticular distance from objects, such as furniture or walls, such asinstances wherein the robotic device follows along the perimeters orwalls of the environment. In other instances, distance sensors may beused to verify the location of objects, such as obstacles andperimeters, in a virtual representation, such as a map, of theenvironment. Distances to objects measured by the distance sensor arecompared to corresponding distances to objects in the virtualrepresentation of the environment. For example, in some embodiments, aninitial map of the environment may contain a perimeter in a particularlocation, which upon verification of the perimeters using a distancesensor may not be found to be in that particular location. In FIG. 6A,for example, initial map 600 comprises perimeter segment 601 extendingfrom dashed line 602 to dashed line 603 and perimeter segment 604extending from dashed line 605 to 606, among the other segments combinedto form the entire perimeter shown. Based on initial map 600 of theenvironment, route plan 607 covering central areas of the environmentmay be devised and executed for cleaning. Upon completion of route plan607, the robotic device may cover the perimeters for cleaning whilesimultaneously verifying the mapped perimeters using at least onedistance sensor of the robotic device, beginning at location 608 in FIG.6B. As the robot follows along the perimeter, area 609 beyond previouslymapped perimeter segment 601 is discovered. This may occur if, forexample, a door in the location of perimeter segment 601 was closedduring initial mapping of the environment. Newly discovered area 609 maythen be covered by the robotic device as is shown in FIG. 6B, afterwhich the robot may return to following along the perimeter. As therobot continues to follow along the perimeter, area 610 beyondpreviously mapped perimeter segment 604 is discovered. This may occurif, for example, a curtain in the location of perimeter segment 604 isdrawn shut during initial mapping of the environment or other obstaclesare removed. Newly discovered area 610 may then be covered by therobotic device as is shown in FIG. 6B, after which the robot may returnto following along the perimeter until reaching an end point 611. Inembodiments, the newly discovered areas may be stored in a second mapseparate from the initial map.

In embodiments, the outcome of a work session (e.g., a cleaning sessionbetween visits to a base station where a robot is charged) is stored inmemory (e.g., of the robot, base station, or remote computer) and may beused for analysis (e.g., by a process of the robot, base station, orremote computer). Saved outcomes from past iterations may be used toinfluence the future real-time route planning whereby analysis ofhistoric data may aid in formulating better route plans. In embodiments,the consecutively executed route plans completed within correspondingconsecutively discovered areas in one work session are adjusted overtime such that repeat coverage and coverage time are minimized over aperiod of robotic device's life. U.S. Patent App. No. 62/666,266, thecontents of which are hereby incorporated by reference, demonstratessuch an approach where coverage by the robotic device becomes moreefficient over time through adjustments applied to route plans overtime. In some embodiments, the order of covering discovered areas withinroute plans may also be optimized based on historical data. For example,the order of coverage of discovered areas which yields highest rewardbased on an MDP algorithm may indicate an optimized order of coveringdiscovered areas. This process of optimization, over time, organicallyleads to an order which is most efficient. For example, covering alldiscovered areas within one room, then moving onto discovered areaswithin a second room is more efficient than covering some discoveredareas in one room, covering some discovered areas in a second room,returning back to the first room to cover the remaining areas and thenback to the second room to cover remaining areas.

In some embodiments, to improve navigation of the robotic device whileexecuting a path plan, the processor of the robotic device predicts orinfers the position of the robotic device. To do so, in someembodiments, a processor of the robotic device minimizes error, e.g.,the difference between the true position and predicted position. Theposition of the robotic device may be predicted given a previousposition estimate x_(t), sensor measurements u_(t) from, for example,wheel encoders, and time step Δt:

{circumflex over (x)} _(t+1)=model(x _(t) ,u _(t) ,Δt)

wherein {circumflex over (x)}_(t+1) is the new estimated position. Atsome time points the pose of the robotic device is received from, forexample, sensor pose updates, the corrected pose denoted by x_(t+1) attime t+1. Due to wheel slippage, measurement noise, etc., a trackingerror e=x_(t+1)−{circumflex over (x)}_(t+1) exists.

To reduce the tracking error, and the performance of the positionprediction algorithm, parameters may be introduced into the model. Forexample, in embodiments, a parameterized model {circumflex over(x)}_(t+1)=model (x_(t),p_(t)) may be used, with observations x_(t) andunknown parameters p_(t) at time t. The parameters p_(t) which minimizethe prediction error x_(t+1)−{circumflex over (x)}_(t+1), whereinx_(t+1) is the true observations and {circumflex over (x)}_(t+1) may beused as the predicted observations at time t+1. In embodiments,recursive estimation may be used to estimate the parameters and may beformally represented by:

p _(t+1) =p _(t) +K _(t+1)(x _(t+1) −{circumflex over (x)} _(t+1))

wherein p_(t+1) is the estimated parameter at time t+1. K_(t+1) is thegain and may be used to determine the degree of influence the predictionerror x_(t+1)−x _(t+1) has on the updated parameter estimate p_(t+1).The parameter may be updated in the direction of the error gradient,i.e.,

K _(t+1) =Q _(t+1)Ψ_(t+1)

wherein

$\Psi_{t + 1}^{T} = \frac{\partial{\hat{x}}_{t + 1}}{\partial p_{t}}$

is the gradient and Q_(t+1) may be suitably chosen depending on the typeof algorithm desired. For example, for tracking error gradient descentQ_(t+1) may be a constant or for tracking error gradient descent withnormalized gradient Q_(t+1) may be equal to

$\frac{\gamma}{{\Psi_{t + 1}}^{2} + {bias}}$

or for Kalman filter K_(t+1), Q_(t+1) may be considered Kalman gain.

In embodiments, the model may include parameters p_(t)=[p₁, . . . ,p_(N)], whereby the position prediction model becomes a function ofparameters p_(t) as well:

x _(t+1)=model(x _(t) ,u _(t) ,Δt,p _(t))

These wheel parameters may be, for example, wheel radii, distancebetween wheels, wheel position, sensor position, etc. As describedabove, recursive estimation may be used to estimate these parametersusing p_(t+1)=p_(t)+K_(t+1)(x_(t+1)−x ₊₁), where K_(t+1)=Q_(t+1)Ψ_(t+1)and

$\Psi_{t + 1}^{T} = {\frac{\partial{\hat{x}}_{t + 1}}{\partial p_{t}}.}$

In embodiments, tracking error gradient descent approach may be usedwhereby p is incrementally perturbed in the direction that reduces theerror the most, for example in the direction of the descent on thetracking error function E=½e^(T)e. This results in the following updaterule for p:

$p_{t + 1} = {p_{t} + {h\frac{\partial{\hat{x}}_{t + 1}^{T}}{\partial p_{t}}e}}$

where Q_(t+1)=h, a small step size and e=(x_(t+1)−x _(t+1)). This methodhas several hyperparameters that the processor of the robotic device mayuse for fine tuning. For example, an adaptive step size h(t) may be usedwhere for example it may decay with time or Q may be pre-multiplied bysome matrix M, such that updates are larger in directions with highcertainty and smaller for areas with low certainty or stabilityimprovements such as outlier detection may be used. Thesehyperparameters are merely an example and are not intended to limithyperparameters that may be used with this method (which is not tosuggest other statements are limiting).

In some embodiment, where an Extended Kalman Filter is used, the statespace may be augmented to:

${\overset{\sim}{x}}_{t} = \begin{pmatrix}x_{t} \\p_{t}\end{pmatrix}$

and the Kalman filter applied to {tilde over (x)}. Assuming parametersare time-invariant, zero noise is modeled on the parameter part of thestate and the state space can be represented as below.

${\overset{\sim}{x}}_{t + 1} = {{\overset{\sim}{\Phi}(\; \ldots \;)} = \begin{pmatrix}{{model}\left( {x_{t},u_{t},{\Delta \; t},p_{t}} \right)} \\p_{t}\end{pmatrix}}$

The Jacobian matrix for the augmented state space is then given by:

$\overset{\sim}{F} = {\frac{\partial\overset{\sim}{\Phi}}{\partial{\overset{\sim}{x}}_{t}} = \begin{pmatrix}F & \Psi_{t}^{T} \\0 & I\end{pmatrix}}$

where F is the Jacobian from the non-augmented Kalman process.

For example, a robot's motion may be modeled by the equations below:

{dot over (x)}=ν cos ω

{dot over (y)}=ν sin ω

{dot over (θ)}=ω

where ν and ω are translational and rotational velocities, respectively.As previously described, the navigation strategy of the robot withdifferential drive is to move in a straight line, rotate in place, thenmove in a straight line again. For such motion, a forward model may beused to determine the pose of the robotic device with the same equationsas previously described:

x _(t+1) =x _(t) +νΔt cos θ_(t)

y _(t+1) =y _(t) +νΔt sin θ_(t)

θ_(t+1)=θ_(t) +ωΔt

where translational and rotational velocities of the robot are computedusing observed wheel angular velocities ω_(l) and ω_(r).

$\begin{pmatrix}v \\w\end{pmatrix} = {{J\begin{pmatrix}\omega_{l} \\\omega_{r}\end{pmatrix}} = \begin{pmatrix}{r_{l}/2} & {r_{r}/2} \\{{- r_{l}}/l} & {r_{r}/l}\end{pmatrix}}$

r₁ and r_(r) represent left and right wheel radii, respectively and l isthe distance between the two wheels, or otherwise the length of the axleconnecting the wheels. The equation for ν, assuming each wheel isrolling, is given by

$v = \frac{\omega \; r}{2}$

and the equation for ω stems from

${{\omega \left( {R + \frac{l}{2}} \right)} = {{v_{r}\mspace{14mu} {and}\mspace{14mu} {\omega \left( {R - \frac{l}{2}} \right)}} = v_{l}}},$

which may be combined to form the equation

$\omega = {\frac{v_{r} - v_{l}}{2}.}$

Here, wheel sizes are considered parameters p_(t)=[r_(i),r_(r)]^(T) andJ=J(p_(t)). For observed left and right wheel rotational velocitiesu_(t)=[ω_(l),ω_(r)]^(T), parameters p are estimated to improve theforward model. Here the state vector is abbreviated by:

$x_{t} = \begin{pmatrix}x_{t} \\y_{t} \\\theta_{t}\end{pmatrix}$

resulting in the model below.

${\hat{x}}_{t + 1} = {{{model}\left( {x_{t},u_{t},{\Delta \; t},p_{t}} \right)} = \begin{pmatrix}{x_{t} + {v_{t}\Delta \; t\; \cos \; \theta_{t}}} \\{y_{t} + {v_{t}\Delta \; t\; \sin \; \theta_{t}}} \\{\theta_{t} + {\omega_{t}\Delta \; t}}\end{pmatrix}}$

with,

$\begin{pmatrix}v_{t} \\\omega_{t}\end{pmatrix} = {{J\left( p_{t} \right)}u_{t}}$

Using the model given for {circumflex over (x)}_(t+1) and the equationsfor velocities ν and ω, the gradient in direction of the modelparameters can be computed as below at a time point t:

$\frac{\partial{\hat{x}}_{t + 1}}{\partial p_{t}} = {\frac{1}{2}\Delta \; {t\begin{pmatrix}{\omega_{l}\cos \; \theta_{t}} & {\omega_{r}\cos \; \theta_{t}} \\{\omega_{l}\sin \; \theta_{t}} & {\omega_{r}\sin \; \theta_{t}} \\\frac{{- 2}\; \omega_{l}}{b} & \frac{2\; \omega_{r}}{b}\end{pmatrix}}}$

The computed gradient may be used when calculating updated parametersusing, for example, the equation previously described for tracking errorgradient descent:

$p_{t + 1} = {p_{t} + {h\frac{\partial{\hat{x}}_{t + 1}^{T}}{\partial p_{t}}e}}$

or in the Jacobian matrix for an augmented state space when using an EKFapproach described earlier:

$\overset{\sim}{F} = {\frac{\partial\overset{\sim}{\Phi}}{\partial{\overset{\sim}{x}}_{t}} = \begin{pmatrix}F & \Psi_{t}^{T} \\0 & I\end{pmatrix}}$

where

$\Psi_{t + 1}^{T} = \frac{\partial{\hat{x}}_{t + 1}}{\partial p_{t}}$

and F is the Jacobian from the non-augmented Kalman process.

FIGS. 7A-7E illustrate estimated left and right wheel radii modelparameters using tracing error gradient decent and augmented Kalmanfilter methods given simulated data with Gaussian noise. The averageerror in the pose is also illustrated. In FIG. 7A, augmented Kalmanfilter (l_aug, r_aug) and tracing error gradient descent (l_desc,r_desc) estimating left and right wheel radii are illustrated in graph700. This is plotted along the y-axis as a wheel ratio of the predictedwheel radius/expected wheel radius (e.g. as reported by themanufacturer) at time steps of 0.05 seconds along the x-axis. Groundtruth (gt_left, gt_right) of actual wheel radius/expected wheel radiusfor the left and right wheel is illustrated and is what both modelsattempt to predict. Graph 701 illustrates the error of the predictedpose, calculated as the norm of the difference between the predictedpose and actual pose for augmented Kalman filter, tracing error gradientdescent, and for comparison, a vanilla EKF at time steps of 0.5 secondsalong the x-axis. This demonstrates that improving model accuracy usingaugmented Kalman filter or tracing error gradient descent lowers theprediction error significantly. In FIG. 7B, a lower step size in thetracing error gradient descent method is used, in comparison to FIG. 7A,which is evident as ground truth is approached much slower in graph 702and is reflected in the prediction error in graph 703. In FIG. 7C highernoise for the parameters is used in comparison to FIG. 7A. This has amost noticeable effect on the augmented Kalman filter method, producingmuch noisier estimates in graph 704. However, this appears to have noeffect on the error shown in graph 705. In FIG. 7D time variant wheelsizes are used in comparison to FIG. 7A. Both augmented Kalman filterand tracing error gradient descent methods are capable of trackingchanges in parameters over time as is shown in graph 706 and error inpose over time is almost unchanged in graph 707. In FIG. 7E, a drop inthe parameters is implemented (e.g. due to deformation) in comparison toFIG. 7A. It can be seen that both methods quickly adapt in graph 708 andthat the error in the pose in graph 709 only jumps slightly for a shortperiod of time after encountering the drop in parameters.

A feedforward neural network may be used to learn a motion model for therobot given sufficient training data. For example, a model may considerthe old position and measured velocity to estimate the new position ofthe robot. This model would be capable of mapping coordinate dependentparameters, such as a tilted floor. As a further example, a model mayonly consider the measured velocity of the robot having an initialstarting position of (0, 0, 0). While such models may be trained to ahigh degree of accuracy for predicting the position of the robot, theydo not take previous measurements into account and online calibrationwould require frequent backpropagation of errors, potentially resultingin substantial computational costs. Therefore, in embodiments, arecurrent neural network (RNN) may be used for predicting the positionof the robot using a trained model. In contrast to classical feedforwardneural networks, RNNs consider both the current input and what wasperceived in previous steps, an approach which is observed in theposition prediction methods described earlier. Internally, recurrentunits have a hidden internal state which influences the output. Thishidden state is updated with each input, providing a feedback loop atevery time step in the input series. Different hierarchies may be used,for example, wherein output from one time step is fed back as input atthe next time step. FIGS. 8A-8C illustrate example schematics forposition prediction with implemented recurrent units. In FIG. 8A,recurrent unit 800 is the state box. Since recurrent units have memory,arrow 801 represents the ability of the recurrent unit to remember itsstate. Recurrent unit 800 allows previous states to be considered whenpredicting the new state of the robotic device. The remaining schematicis a classic neural network. In FIG. 8B, recurrent units 802 and 803 areused as a pre-filter for odometer and inertial measurement unit (imu)data. The output of the pre-filter is used in a ground truth (GT) modelto predict the new state of the robotic device. In FIG. 8C, recurrentunits 804 and 805 are used to learn mapping observations to parameters pof a forward/EKF model used to predict the new state of the roboticdevice. Recurrent units in FIGS. 8B and 8C have arrows similar to arrow801 of recurrent unit 800 in FIG. 8A as recurrent units consider whatwas perceived (or observed or measured) in previous steps.

FIG. 9 depicts an example of a robotic device 900 with processor 901,memory 902, sensor 903 and actuator 904. In some embodiments, the robotmay include the features of a robot described herein. In someembodiments, program code stored in the memory 902 and executed by theprocessor 901 may effectuate the operations described herein.

FIG. 10 illustrates a flowchart describing embodiments of a pathplanning method of a robotic device 1000, 1001, 1002, and 1003corresponding with steps performed in some embodiments.

In block diagrams, illustrated components are depicted as discretefunctional blocks, but embodiments are not limited to systems in whichthe functionality described herein is organized as illustrated. Thefunctionality provided by each of the components may be provided bysoftware or hardware modules that are differently organized than ispresently depicted, for example such software or hardware may beintermingled, conjoined, replicated, broken up, distributed (e.g. withina data center or geographically), or otherwise differently organized.The functionality described herein may be provided by one or moreprocessors of one or more computers executing code stored on a tangible,non-transitory, machine readable medium. In some cases, notwithstandinguse of the singular term “medium,” the instructions may be distributedon different storage devices associated with different computingdevices, for instance, with each computing device having a differentsubset of the instructions, an implementation consistent with usage ofthe singular term “medium” herein. In some cases, third party contentdelivery networks may host some or all of the information conveyed overnetworks, in which case, to the extent information (e.g., content) issaid to be supplied or otherwise provided, the information may providedby sending instructions to retrieve that information from a contentdelivery network.

The reader should appreciate that the present application describesseveral independently useful techniques. Rather than separating thosetechniques into multiple isolated patent applications, applicants havegrouped these techniques into a single document because their relatedsubject matter lends itself to economies in the application process. Butthe distinct advantages and aspects of such techniques should not beconflated. In some cases, embodiments address all of the deficienciesnoted herein, but it should be understood that the techniques areindependently useful, and some embodiments address only a subset of suchproblems or offer other, unmentioned benefits that will be apparent tothose of skill in the art reviewing the present disclosure. Due to costsconstraints, some techniques disclosed herein may not be presentlyclaimed and may be claimed in later filings, such as continuationapplications or by amending the present claims. Similarly, due to spaceconstraints, neither the Abstract nor the Summary of the Inventionsections of the present document should be taken as containing acomprehensive listing of all such techniques or all aspects of suchtechniques.

It should be understood that the description and the drawings are notintended to limit the present techniques to the particular formdisclosed, but to the contrary, the intention is to cover allmodifications, equivalents, and alternatives falling within the spiritand scope of the present techniques as defined by the appended claims.Further modifications and alternative embodiments of various aspects ofthe techniques will be apparent to those skilled in the art in view ofthis description. Accordingly, this description and the drawings are tobe construed as illustrative only and are for the purpose of teachingthose skilled in the art the general manner of carrying out the presenttechniques. It is to be understood that the forms of the presenttechniques shown and described herein are to be taken as examples ofembodiments. Elements and materials may be substituted for thoseillustrated and described herein, parts and processes may be reversed oromitted, and certain features of the present techniques may be utilizedindependently, all as would be apparent to one skilled in the art afterhaving the benefit of this description of the present techniques.Changes may be made in the elements described herein without departingfrom the spirit and scope of the present techniques as described in thefollowing claims. Headings used herein are for organizational purposesonly and are not meant to be used to limit the scope of the description.

As used throughout this application, the word “may” is used in apermissive sense (i.e., meaning having the potential to), rather thanthe mandatory sense (i.e., meaning must). The words “include”,“including”, and “includes” and the like mean including, but not limitedto. As used throughout this application, the singular forms “a,” “an,”and “the” include plural referents unless the content explicitlyindicates otherwise. Thus, for example, reference to “an element” or “aelement” includes a combination of two or more elements, notwithstandinguse of other terms and phrases for one or more elements, such as “one ormore.” The term “or” is, unless indicated otherwise, non-exclusive,i.e., encompassing both “and” and “or.” Terms describing conditionalrelationships, e.g., “in response to X, Y,” “upon X, Y,”, “if X, Y,”“when X, Y,” and the like, encompass causal relationships in which theantecedent is a necessary causal condition, the antecedent is asufficient causal condition, or the antecedent is a contributory causalcondition of the consequent, e.g., “state X occurs upon condition Yobtaining” is generic to “X occurs solely upon Y” and “X occurs upon Yand Z.” Such conditional relationships are not limited to consequencesthat instantly follow the antecedent obtaining, as some consequences maybe delayed, and in conditional statements, antecedents are connected totheir consequents, e.g., the antecedent is relevant to the likelihood ofthe consequent occurring. Statements in which a plurality of attributesor functions are mapped to a plurality of objects (e.g., one or moreprocessors performing steps A, B, C, and D) encompasses both all suchattributes or functions being mapped to all such objects and subsets ofthe attributes or functions being mapped to subsets of the attributes orfunctions (e.g., both all processors each performing steps A-D, and acase in which processor 1 performs step A, processor 2 performs step Band part of step C, and processor 3 performs part of step C and step D),unless otherwise indicated. Further, unless otherwise indicated,statements that one value or action is “based on” another condition orvalue encompass both instances in which the condition or value is thesole factor and instances in which the condition or value is one factoramong a plurality of factors. Unless otherwise indicated, statementsthat “each” instance of some collection have some property should not beread to exclude cases where some otherwise identical or similar membersof a larger collection do not have the property, i.e., each does notnecessarily mean each and every. Limitations as to sequence of recitedsteps should not be read into the claims unless explicitly specified,e.g., with explicit language like “after performing X, performing Y,” incontrast to statements that might be improperly argued to imply sequencelimitations, like “performing X on items, performing Y on the X'editems,” used for purposes of making claims more readable rather thanspecifying sequence. Statements referring to “at least Z of A, B, andC,” and the like (e.g., “at least Z of A, B, or C”), refer to at least Zof the listed categories (A, B, and C) and do not require at least Zunits in each category. Unless specifically stated otherwise, asapparent from the discussion, it is appreciated that throughout thisspecification discussions utilizing terms such as “processing,”“computing,” “calculating,” “determining” or the like refer to actionsor processes of a specific apparatus, such as a special purpose computeror a similar special purpose electronic processing/computing device.Features described with reference to geometric constructs, like“parallel,” “perpendicular/orthogonal,” “square”, “cylindrical,” and thelike, should be construed as encompassing items that substantiallyembody the properties of the geometric construct, e.g., reference to“parallel” surfaces encompasses substantially parallel surfaces. Thepermitted range of deviation from Platonic ideals of these geometricconstructs is to be determined with reference to ranges in thespecification, and where such ranges are not stated, with reference toindustry norms in the field of use, and where such ranges are notdefined, with reference to industry norms in the field of manufacturingof the designated feature, and where such ranges are not defined,features substantially embodying a geometric construct should beconstrued to include those features within 15% of the definingattributes of that geometric construct.

I claim:
 1. A robot-implemented, real-time, method of coverage pathplanning, the method comprising: obtaining, with one or more processorsof a robot, environment-sensor data indicating distances from the robotto surfaces in a portion of a working environment of the robot fromsensors carried by the robot; obtaining, with one or more processors ofthe robot, odometry-sensor data indicating changes in position of therobot over time; based on the environment-sensor data and theodometry-sensor data, determining in real-time, with one or moreprocessors of the robot, at least a part of a coverage path of the robotthrough the working environment, wherein determining the at least partof the coverage path comprises determining lengths of segments of thecoverage path, the segments having a linear motion trajectory, and thesegments forming a boustrophedon pattern that covers at least part ofthe working environment; and commanding, with one or more processors ofthe robot, an electric-motor driver to move the robot along the at leastpart of the path.
 2. The method of claim 1, wherein: the robot comprisesa plurality of sensors from which the environment-sensor data isobtained; and at least a plurality of the plurality of sensors outputdata that is independent and identically distributed.
 3. The method ofclaim 1, wherein: the working environment is divided into a plurality ofregions in a region graph; a same instance of the boustrophedon patternis repeated in each region in a region-specific orientation determinedfor the respective region.
 4. The method of claim 1, wherein: theboustrophedon pattern comprises at least four segments with motiontrajectories in alternating directions.
 5. The method of claim 1,wherein: the coverage path includes an edge linked by vertices at whichthe coverage path is re-evaluated by the one or more processors of therobot.
 6. The method of claim 1, comprising: in response to thecommanding, calculating and sending a plurality of electric pulses tothe electric motor such that the movement of the robot is in straightline, the length of the line being determined in real-time and based onthe environment-sensor data and the odometry-sensor data.
 7. The methodof claim 1, wherein: determining at least part of the coverage pathcomprises segmenting the working environment into a central area and amarginal area; and the coverage path specifies that the central area isvisited by the robot before the marginal area.
 8. The method of claim 1,wherein: the working environment is segmented, by one or more processorsof the robot or a remote computing system, into regions; a preliminarypath is determined before beginning to traverse the working environmentand not in real time; the preliminary path includes a different instanceof the pattern in each of the regions; and at least some instances ofthe pattern are adjusted in real-time by the determining the at leastpart of the path while traversing a respective one of the regions. 9.The method of claim 1, wherein: determining at least part of thecoverage path comprises selecting motion trajectories along segments ofthe path before beginning to traverse the respective segments.
 10. Themethod of claim 1, wherein: attributes of segments are selected with aMarkov Decision Process.
 11. The method of claim 1, wherein: determiningat least part of the coverage path comprises optimizing an objectivefunction that assigns values to at least two of the following: turns,repeated coverage, transitions between different types of flooring, orswitching rooms, and assigns a reward based on the values.
 12. Themethod of claim 11, wherein: determining at least part of the coveragepath comprises minimizing a cost function or maximizing a fitnessfunction over a plurality of segments of the path.
 13. The method ofclaim 11, comprising: executing a plurality of tasks with the robot,each task including covering the working environment; and adjusting astate-action value function or policy between at least some of the tasksto further optimize the object function.
 14. The method of claim 11,wherein: determining at least part of the coverage path comprises usinga net cumulative reward to evaluate the coverage path and configuringsegments predicted to result in more optimal net reward thanalternatives and avoiding segments that previously resulted insub-optimal net reward.
 15. The method of claim 1, wherein; the robot isa floor cleaning robot comprising: a battery; a cleaning tool; theelectric-motor controller; and two or more electric motors coupled tothe battery and controlled by the motor controller.
 16. The method ofclaim 1, wherein: a position of the robot is determined based on thedata indicating changes in position of the robot over time and theoutput of a Kalman filter.
 17. A method, comprising: cleaning a firstportion of a room with a robot traversing a first pattern, wherein: atleast part of the first pattern is determined in real-time by the robotbased on sensor data obtained by the robot while traversing the firstpattern and an objective function that assigns values to at least one ofthe following: turns, repeated coverage, transitions between differenttypes of flooring, or switching rooms; and cleaning a second portion ofthe room by traversing a perimeter of the room with the robot.
 18. Themethod of claim 17, wherein: the first portion is cleaned before thesecond portion; and the first pattern comprises a zig-zag pattern, aboustrophedon pattern, a spiral pattern, a Gosper curve, a Morton curve,a Hilbert curve, or Archimedean chords.
 19. A device, comprising: arobot comprising: a battery; an electric-motor controller; two or moreelectric motors coupled to the battery and controlled by the motorcontroller; one or more processors configured to control theelectric-motor controller; and one or more tangible, non-transitory,machine readable media storing instructions that when executed by atleast some of the processors effectuate operations comprising:obtaining, with one or more processors of a robot, environment-sensordata indicating distances from the robot to surfaces in a portion of aworking environment of the robot from sensors carried by the robot;obtaining, with one or more processors of the robot, odometry-sensordata indicating changes in position of the robot over time; based on theenvironment-sensor data and the odometry-sensor data, determining inreal-time, with one or more processors of the robot, at least a part ofa coverage path of the robot through the working environment, whereindetermining the at least part of the coverage path comprises determininglengths of segments of the coverage path, the segments having a linearmotion trajectory, and the segments forming a boustrophedon pattern thatcovers at least part of the working environment; and commanding, withone or more processors of the robot, an electric-motor driver to movethe robot along the at least part of the path.
 20. The device of claim19, wherein: the robot comprises a plurality of sensors from which theenvironment-sensor data is obtained; and at least a plurality of theplurality of sensors output data that is independent and identicallydistributed.
 21. The device of claim 19, wherein: the work environmentis divided into a plurality of regions in a region graph; the pattern isrepeated in each region in a region-specific orientation determined forthe respective region.
 22. The device of claim 19, wherein: the patterncomprises a boustrophedon pattern with at least four segments withmotion trajectories in alternating directions.
 23. The device of claim19, wherein: the coverage path includes a line linked by vertices atwhich the coverage path is re-evaluated by the one or more processors ofthe robot.
 24. The device of claim 19, the operations comprising: inresponse to the commanding, calculating and sending a plurality ofelectric pulses to the electric motor such that the movement of therobot is in straight line, the length of the line being determined inreal-time and based on the environment-sensor data and theodometry-sensor data.
 25. The device of claim 19, wherein: determiningat least part of the coverage path comprises segmenting the workingenvironment into a central area and a marginal area; and the coveragepath specifies that the central area is visited by the robot before themarginal area.
 26. The device of claim 19, wherein: the workingenvironment is segmented, by one or more processors of the robot or aremote computing system, into regions; a preliminary path is determinedbefore beginning to traverse the working environment, wherein traversingthe working environment is based on the preliminary path; thepreliminary path includes a different instance of the pattern in each ofthe regions; and at least some instances of the pattern are adjusted inreal-time by the determining the at least part of the path whiletraversing a respective one of the regions.
 27. The device of claim 19,wherein: determining at least part of the coverage path comprisesselecting motion trajectories along segments of the path beforebeginning to traverse the respective segments.
 28. The device of claim19, wherein: attributes of segments are selected with a Markov DecisionProcess.
 30. The device of claim 19, wherein: determining at least partof the coverage path comprises optimizing an objective function thatassigns values to at least two of the following: turns, repeatedcoverage, transitions between different types of flooring, or switchingrooms; and determining at least part of the coverage path comprisesminimizing a cost function or maximizing a fitness function over aplurality of segments of the path.